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Malicious Software 



Dear everyone: 
This system is 
Infected! 
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Malicious Software 



I don't like you. 
You are 



annoying. 
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annoying. 
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Malicious Software 



Dear everyone: 
This system is 
A-OK! 



Security Software 
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Malicious Software 



That's what I'm 
talkin' 'bout 
(Bruce) Willis! 



Security Software 
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Malicious Software Security Software 






*scan* 
*scan* 
*scan* 
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Security 
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f 



Malicious Software 



You are 
similarly 
annoying! 



Checkmate 
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Malicious Software Security Software 



Checkmate 






Don't believe me! 
I'm compromised! 



ity Software 
sOK. 
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Malicious Software 



Are you kidding 
me? F*&@ A self- 
checking tricorder... 
This is ridiculous! 




*scribb 
*scribb 
*scribb 
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Malicious Software Security Software 



Checkmate 





l...am...O...K... 




ity Software 
sOK. 



MITRE 



11 

© 2012 The MITRE Corporation. All rights reserved. 



Timing-Based Attestation 
(aka Software-Based Attestation) 

Based on concept of Pioneer by Seshadri et al. 
Assumptions 

- You can know the client hardware profile 

- Your self-check is the most optimized implementation 

Implemented from scratch, independently 
confirmed previous results. 

Source code is released so we can work with 
other researches to validate/improve it. 

http://code.google.eom/p/timing-attestation 
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Nitty Gritty How Does it Work? 



• The self-check is hand coded asm to try to 
build a timing side-channel into its execution 

• The system measurements are things like you 
would fine in any memory integrity checking 

software like MS's PatchGuard, Mandiant's 
MIR, or HBGary's Active Defense. 

• We're going to focus on the self-check, 
because that's what we have that others don't 
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First principles 1 



• "I want to know that my code isn't changed while 
it's running" 

• Malware does this by self-checksumming or even 
self-timing with an rdtsc instruction. This 
commonly detects hardware and software 
breakpoints. 

• Problem: An attacker (from malware's 
perspective the analyst, from our perspective, 
malware) can just force the check to always 
succeed. 
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Original code 



int main(){ 

foo = Selfcheck(); 
jf(foo == 0xl2341234){ 

DoSomething(); 

return SUCCESS; 

} 

else{ 

return FAILURE; 

} 

} 
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Attacker rewrites code 



int main(){ 

foo = So l fchockQ; foo = 0x12341234; 
jf(foo == 0xl2341234){ 

DoSomethingO; 

return SUCCESS; 

} 

else{ 

return FAILURE; 

} 

} 
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First principles 2 



• At this point basically everyone gives up, and 
just goes with code obfuscation. 

• We go with 

- 1) making the self-check a function of a nonce 

- 2) controlling the execution environment to yield 
highly predictable runtime 

- 3) just let the code run, and evaluate whether it 
was tampered with back at a remote server, 
based on the self-checksum AND the runtime 
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New outline for code 



int main(){ 

int selfchecksum[6]; 

nonce = WaitForMeasurementRequestFromVerifierQ; 
Selfcheck(&selfchecksum, nonce); 
SendResultsToVerifier(selfchecksum,nonce); 
results = DoSomething(); 
SendResultsToVerifier(results); 
return SUCCESS; 
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Thoughts on the nonce 



• No single correct value that the attacker can 
send-back to indicate the code is intact 

• Large nonce and/or self-checksum size 
reduces probability of encountering 
precomputation attacks 

- Attacker needs to store 2 A 32*192 bits (96GB) in 
RAM for a 32 bit precomputation or 2 A 64*384 bits 
(768 Zetabytes) for our 64 bit implementation 
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What should we actually read to 
indicate the code is unmodified? 

A pointer which points at our own code 

- We will call this DP for data pointer 

- This indicates the memory range where our code is 
executing from. Original Pioneer assumed it was in a fixed 
location that we could know, but on Widows, no such luck 
(ASLR & faux ASLR) 

Our own code bytes 

- We will call this *DP (C syntax) or [DP] (asm syntax) to 
indicate we're dereferencing the data pointer 

Our instruction pointer (EIP) 

- This also indicates the memory range where our code is 
executing from. Should generally agree with DP. 
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Selfcheck() .01 

void Selfcheck(int * selfchecksum, int nonce){ 
int * DP = GetMyCodeStart(); 
int * end = GetMyCodeEndQ; 
while(DP<end){ 

selfchecksum[0] += nonce; 

selfchecksum[l] += *DP; 

asm{ call $+5; 

pop eax; 

mov EIP, eax;} 
selfchecksum[2] += EIP; 
mix(selfchecksum); 
DP++; 

} 

} 
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Problems with SelfcheckQ .01 



• It's parallelizable. An attacker can add 
compute power from the GPU or any other 
processing we're not using to counteract any 
time he may incur by forging the self- 
checksum 

— We can counter this with "strongly ordered 
function" like A+B0C + D0E + F etc. Because 
the longer the chain, the less likely 

((((A+B)0C)+D)0E)+F)==(A+B)0C+D)0(E+F)for 
instance. 
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Problems with SelfcheckQ .01 



• There is potentially lots of wasted cycles, so 
an attacker may be able to add an if() case 
with no overhead. 

- So we need to handcode assembly, and try to 
make sure it is using as much of the 

microarchitecture components as possible so 
there is no "free" computation available to an 

attacker. Otherwise he can just do... 
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Selfcheck() .01 attack 



void Selfcheck(int * selfchecksum, int nonce){ 
int * DP = GetMyCodeStart(); 
int * end = GetMyCodeEndQ; 
while(DP<end){ 

selfchecksum[0] += nonce; 

if(DP == badbits) selfchecksum[l] += cleanbits; 

else selfchecksum[l] += *DP; 

asm{ call $+5; 



} 



} 



pop eax; 

mov EIP, eax;} 
selfchecksum[2] += EIP; 
mix(selfchecksum); 
DP++; 
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Network Timing Implementation 



Server 



Measurement Type: FOO, 
Nonce = 0xf005ball 



Client 



Self-Checksum, 
Nonce = 0xf005ball 



FOO m 



easurementresujts 




Selfcheck (Nonce = 0xf005ball) 




FOO measurement 
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Network Timing Implementation 

(with attack) 



Server 



Measurement Type: FOO, 
Nonce = 0xf005ball 



Client 



Selfchecksum, 
Nonce = 0xf005ball 



FOO mea 



surement resets 




Selfcheck (Nonce = 0xf005ball) 




FOO measurement 
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One more problem with SelfcheckQ .01 

• Also, notice that EIP will actually always be the 
exact same value each time through the loop. So 
the attacker could create his own checksum 
routine off to the side, which instead of 
calculating EIP, just hardcodes it based on 
wherever the self-check got loaded into memory. 

- We need to make it so that the attacker can't 
hardcode the EIP. We can do this by breaking the self- 
check into multiple blocks, and pseudo-randomly 
picking a different block each time through the loop 
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From A. Seshadri, M. Luk, E. Shi, A. Perrig, L. van Doom, and P. Kh 
Pioneer: verifying code integrity and enforcing untampered code 



osla. 

execution on legacy systems. 




jmp 
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PRNG 



• But now we need a pseudo-random number 
generator, seeded by our nonce. 

• We used the same one Pioneer did: 

• PRN new = PRN current * ( PRN2 C urrent 0R 5 ) 
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New self-check .02 pseudocode 



PrologO; 

BLOCK0_MACRO (expanded) 

if(loopcounter == 0) jmp done; //This used to be our while loop 
loopcounter-; 
add ecx, [esp]; 



//after this ecx (accumulator) = EIP_SRC + EIP_DST 
//ecx = EIP_SRC + EIP_DST XOR PRN 
//ecx = EIP_SRC + EIP_DST XOR PRN + DP 
//ecx = EIP_SRC + EIP_DST XOR PRN + DP XOR [DP] 
//New PRN in each block 
//We pick a new DP based on the PRN 
mix(selfchecksum,ecx); //Rotates checksum by 1 bit to add diffusion 
ecx = blockOBase + (blockSize*(PRN & 3)); //Calc next block based on PRN 
call ecx; //goto next block, EIP_DST in ecx, EIP_SRC on stack 

BLOCKl_MACRO 
BLOCK2 MACRO 



xor ecx, PRN; 
add ecx, DP; 
xor ecx, [DP]; 
updatePRNQ; 
updateDPQ; 



• • • 



BLOCK7_MACRO 
done: 

EpilogQ; 

MITRE 
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add ecx, [es-pj 
add esp, 4 



mov eax> esi 
mul eax 
or eax, 5 
add esL eax 
\or ccs. esi 



add ecx h edi 
\or ccx> [edi] 
mov eax> esi 
I \or edx< edx 
div metnRange 
add edx> codeStart 
mov edi, edx 



mov cax, dr7 
add ecx, eax 
| xor ecx, [esp] 
add esp< 4 



Lest esi, esi 

mov cax, [ebp-i-4" 



mov edSn [ebpj 
mov cax, [edx+4j 
\or cm, esi 
add ecx, eax 



Public releasi 



MIXJtilP 

ELPjiRC ([esp]) + EI1 ] _DST fees) 
ecx is then used as an accumulator 
Etcset stack after fclF SHC push 



UJ > DAitJ ] RN_\0\RO 

Create a copy of x before squaring 



cax = x*x 



cax = (x*x OR 5) 

PRN = x + {x*x OR 5) 

Mix PRN with the accumulator ecx 



READ_AN D_U l*DAl"L_Di J _ VARO 
Mix DP with accumulator ecx 
Mix *DF with accumulator ecx 
Move FRN to eax 
Clear edx 

edx — PRN modulo memKange 
edx=codeStart-KPRN mod memRange) 
Update DP to new value 



READ_LJEt_iiDVm_VARQ 
Copy the DR7 register 
Mix DR7 with accumulator ecx 
Mix HFLAGS with accumulator ecx 
Reset stack after HFLAGS push 



READ_RAND_RET U RN_ADDRESS 
AND l*R.N with self and set flags 
Move PARENT RET to eax 
Hardcoded bytes for ifrPF) jump 6 
l*t is parity flag .set by test esi;, esi 
The jump would land at the next xor 
If not jumped over, 

move the GRANDPARENTJihT to eax 

Xor saved net with PRN 

Mix xored saved ret with accumulator 



ed self-check 





CUE CKSU M_U PDA'J E 


mov eax, ebx 


Copy loop counter to eax 


and eax, 3 


Use bottom 2 bits of loop counter 




to specify which checksum memory 




entry to directly update. 


xor [csp-hcax*4j H ecx 


Xor ehccksum[eax-i-l J, accumulator 




(+1 because checksum [OJ is below esp) 


bt Lesp+OxlOj, 1 


Set carry flag based on LSB 




of checksum 15 J 


rcr [esp-OxOSj^ 1 


Rotate right with carry chccksimi[Oj 


rcr [esp]< I 


Rotate right with carry checksum", I J 


rcr [esp-i-0xQ4] h 1 


Rotate right with carry chccksum[2] 


rcr [esp+ftxO&] K 1 


Rotate right with carry chccksum[3] 


rcr [esp+QxQCj, 1 


Rotate right with carry chec ksuira[4] 


rcr [esp-i-0xl0j h 1 


Rotate right with carry chccksum[5] 




INTERBLOCK TRANSFER 


sub ebx, I 


Decrement loop counter 


test ebx> ebx 


Check if loop counter is 


jz sctRange 


If 0, jump to minicheefcsum switch 


lea edx< address Table 


Otherwise^ prepare to jump 




to next block. Load address of table 




holding start address ot each block 


mov eax> esi 


Copy PRN to eax 


and eax, 7 


Use bottom i bits to decide which 




block to call to next 


mov ecx, [edx+eax*4] 


Move E1I*_DS 1 to ecx 


call ecx 


Call to next bLock 




Implicitly push EJ1 ] _SRC 
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Figure From Pioneer 



Memory 



/////77 



f t f r f r 

V. rune y 



DP 

PC 



(a) No attack, PC 
and DP arc within the 
correct range. 



Attacker gets 
free DP or EIP 

forgery thanks 
to ASLR. 
We had the 
least overhead 
with this attack 




Mai. tunc 



V. Func ^ 
/////7> 



PC 



DP 



(c) Memory copy at- 
tack 2. PC incorrect, 
DP correct , 



MITRE 



opy Attacks 



////// 

/ ] V. func / 

/ / / / / / / < 



Mai, func 

xxxxx 



DP 



PC 



(b) Memory copy at- 
tack 1. PC correct, 
but DP incorrect. 




By definition, 
more overhead 



than (b) or (c). 
Not a good idea. 



{d) Memory copy at- 
tack 3. PC and DP in- 
correct. 
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How it works without attacker 



Verifier 




I 

N 
T 
R 
A 



Original copy of self-checking 

kernel module 



MeasurementRequest 



Nonce = 0xf005ball 



1 
Call 




Send(Selfchecksum) 



Send(BaseVA=OxlOCO) 



4 

call 



Send(Measurement) 



Svstem 



BaseVA 
= 0x1000 




3 

ret 
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Our current fastest PoC attack 

(built into the public released code for easy toggling) 




I 

N 
T 
R 
A 
N 
E 
T 



Original copy of self-checking 

kernel module 



M easurementRequest 



Nonce = 0xf005ball 



BaseVA 
= 0x1000 



Clean copy of complete 
kernel module 




5 

call 



Send(Selfchecksum) 




Send(BaseVA=0x2000 

6 

Send(Measurement) 




VILVFUNC 




system 



ies that system is clean) 




DP/ 

/(free forgery) 



FORGED 
VFUNC EIP 
RANGE 



BaseVA 
= 0x2000 



EVILVFUNC forges EIP 
to be at the right offset 
In the lied-about DP range 




MITRE 



© 2012 The MITRE Corporation. All rights reserved. 



Other tricks 



• Not discussed in depth due to lack of time, see our full paper, the 
related work, and the source code 

• "The stack trick" - if you store part of your self-checksum *below* 
esp, then you can guarantee that if someone causes an interrupt 
during your execution, part of the self-checksum will be destroyed 

• Put PRN into DR7 and read it to prevent cost-free use of hardware 
breakpoints 

• Read parent and grandparent return addresses off the stack, 
otherwise when the self-check is done it will return to attacker 
code (important for TOCTOU as described in a little bit) 

• Additional control flow integrity comes from doing a mini- 
checksum over 3 rd party modules which we depend on, or that we 
indirectly depend on. So if we depend on ntoskrnl.exe and it 
depends on hal.dll, then we measure parts of both. 



MITRE 



35 

© 2012 The MITRE Corporation. All rights reserved. 



Some stuff that's been suggested that we 
tried but ultimately backed away from 

Polymorphic self-check code 

- Because due to the cache misses and branch mispredictions, 
this increases the absolute runtime of the code. Also, the 
attacker can implement a non-polymorphic forgery which is way 
faster thanks to no cache misses (we implemented such an 
attack) 

Exploiting the memory hierarchy by filling instruction and 
data cache to capacity 

- Because unless you have sufficient unique order of inclusion into 
self-checksum block variants to fill the cache, the attacker can 
avoid cache spillage by just making his attack have a lx copy of 
each of your unique blocks, and then keeping track of the order 
that the blocks would execute in (we implemented such an 
attack) 



MITRE 



36 

© 2012 The MITRE Corporation. All rights reserved. 



So what are the new results? 



Countered some previous attacks (Castelluccia et al.) and some new 
ones we came up with 

- Implementation lessons learned and design decisions will be 
documented in a future journal paper. 

Demonstrated that the system can work without being NIC-specific 
(Pioneer was built into an open source NIC driver.) 

Showed that it can work over 10 network links of a production 
enterprise LAN (Pioneer said it worked over "same ethernet 
segment") 

Benchmarked the attestation to see the effects on network 
throughput, filesystem read/write performance, and CPU 
benchmarking applications 

Made the first implementation for TPM-based timing-based 
attestation (Schellekens et al. proposed it but didn't implement 
anything.) 

Defined the relation of TOCTOU to existing and new attacks so 
defenses can be better researched. 
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Network Topology 



Server 



Links to client or server are copper. 
All other links are fiber. 




Switch 




Switch 



Client 
1 link 



Client 

2 links 

Client 

3 links 




Switchl^ffl Switch^ Switch 




Client 
10 links 



Client 
8 links 





Switch 



Switch 



Router 




(building 2) 




Router 



Router 



(buildingl) (Core) 
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Can we detect the reference attacker over the 
maximum hop count on our Virginia campus? 
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Can we detect the reference attacker over the 
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Can we detect the reference attacker over the 



maximum hop count on our Virginia campus! 
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Can we detect the reference attacker over the 
maximum hop count on our Virginia campus? 
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Can we use a single bound for measurement 

times anywhere on our network? 
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Can we use a single bound for measurement 

times anywhere on our network? 
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Server 



Trusted Platform Module (TPM) 

Timing Implementation 



TPM Tickstamp 
Nonce = 0xf005b 



Client 



TPM 



all 



Signed Tickstamp 1 & 2 
Se lf-Checksum 
Nonce = 0xf005ball 
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TPM Implementation - Single Hos 
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TPM Implementation - Single Hos 
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TPM Implementation - 32 Hosts 
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TOCTOU 




Attacker moves out of the way, just in time 
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Conditions for TOCTOU 



• 1) The attacker must know when the 
measurement is about to start. 

• 2) The attacker must have some un-measured 
location to hide in for the duration of the 
measurement. 

• 3) The attacker must be able to reinstall as 
soon as possible after the measurement has 
finished. 
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Malicious Software Security Software 



Checkmate 





l...am...O...K... 




ity Software 
sOK. 
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lalicious Software 



Oh, you're about to 
do a self-check? Let 



me just... 



Checkmate 




*erase* 
*erase* 
*erase* 



*erase* 
*erase* 
*erase* 
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Malicious Software Security Software 



Checkmate 





I'm OK 




Security Software 
is OK. 
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Malicious Software 



Done? Good. Let me 
just... 



Checkmate 




*scribble* 
*scribble* 
*scribble* 



*scribble* 
*scribble* 
*scribble* 
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What regal clothes you have, Emperor 

Most software's TOCTOU defense is just assuming it away. 

- Violate our assumption that the attacker can get to the same level 
as the security software, and then for instance pull the 
measurement agent out to a VMM for instance. Then maybe the 
attacker can't see a measurement is about to start. If the attacker 
can get to the VMM, same problem. 

- In the phone/embedded systems realm (FatSkunk/SWATT) they 
have tried to measure the full contents of RAM to implicitly 
counter TOCTOU condition 2. But that's not really practical for PCs 

due to the amount of time necessary, and the "measure all" is of 
dubious utility. (How do you validate that a chunk of heap 

containing code of function pointers is the "correct" value?) 

Control flow integrity violation serves as an enabler for 
TOCTOU attacks 
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Questions? 

{xkovah,ckallenberg} at mitre.org 
http://code.google.eom/p/timing-attestation 
P.s. http://OpenSecurityTraining.info 

- x86 assembly/architecture & rootkits classes (Xeno) 

- Exploits classes (Corey) 

- TPM class (Ariel) 

- VT-x class (David) 

- Intro RE/Malware Static Analysis classes (Matt & Frank) 

- And many others 
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Backup slides 
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Where else has this been used? 



Embedded systems 



(A. Seshadri, A. Perrig, L. van Doom, and P. Khosla. SWATT: Software- 



& wireless sensors 



based attestation for embedded devices) t (M. Shaneck, K. Mahadevan, 

V. Kher, and Y. Kim. Remote software-based attestation for wireless sensors, Y. Choi, J. Kang, and D. Nyang. 
Proactive code verification protocol in wireless sensor network.) 



SCADA 



(A. Shah, A. Perrig, and B. Sinopoli. Mechanisms to provide integrity in SCADA and PCS devices) 



Keyboards to counter BlackHat talk i (Y. Li, J. M. McCune, and A. 

Perrig. SBAP: Software-Based Attestation for Peripherals.) 

Android Phones (M. Jakobsson and K.-A. Johansson. Practical and secure software-based 

attestation.) 
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Future Work 

(Stop trying to hit me, and hit me!) 



• Use analysis-timing-constrained control flow, e.g 
TEAS by Garay & Huelsbergen, to combat 
TOCTOU condition 1 

• Use multiple processors in parallel to combat 
TOCTOU condition 3 

Self-check 1 

Processor 1 ' 



Processor 2 « Self - check 2 



Self-check n 

Processor n • 



Time 



• Investigate timing-based attestation lower level in the 
system (e.g. BIOS & SMM) 
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Who we would like to hear from 



• All of you - How can we build better attacks 
against our PoC implementation? How can we 
combat TOCTOU in a more generic way? 

• Intel/AMD - How can we further optimize our 
assembly? 

• Microsoft - Is there anything we should be 
doing with our NDIS driver to optimize it? 

Could you using timing-based attestation to 
detect PatchGuard being disabled? 
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Proxy Attacks 



Server 



Measurement Type: FOO, 
Nonce = 0xf005ball 



Compromised 
Client 



Faster Client 



Self-Checksum, 
Nonce = 0xf005ball 



Measurement Type: FOO, 
Nonce = 0xf005ball 

Self-Checksum, 
Nonce = OxfOOSball^ 



Self-Check 

(Nonce = 0xf005ball) 



MITRE 



61 



© 2012 The MITRE Corporation. All rights reserved. 



TPM Timing Implementation Proxy Attack 



Server Slow Client TPM Fast Client 
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